MRI reconstruction using deep learning, generative adversarial network and acquisition signal model

ABSTRACT

A method for diagnostic imaging includes measuring undersampled data y with a diagnostic imaging apparatus; linearly transforming the undersampled data y to obtain an initial image estimate {tilde over (x)}; applying the initial image estimate {tilde over (x)} as input to a generator network to obtain an aliasing artifact-reduced image x̆ as output of the generator network, where the aliasing artifact-reduced image x̆ is a projection onto a manifold of realistic images of the initial image estimate {tilde over (x)}; and performing an acquisition signal model projection of the aliasing artifact-reduced x̆ onto a space of consistent images to obtain a reconstructed image {circumflex over (x)} having suppressed image artifacts.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplication 62/678,663 filed May, 31, 2018, which is incorporated hereinby reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

None.

FIELD OF THE INVENTION

The present invention relates generally to magnetic resonance imaging(MRI). More specifically, it relates to techniques for MRIreconstruction.

BACKGROUND OF THE INVENTION

Due to its superb soft tissue contrast, magnetic resonance imaging (MRI)is a major imaging modality in clinical practice. MRI imagereconstruction is typically an ill-posed linear inverse problemdemanding time and resource intensive computations that substantiallytrade off accuracy for speed in real-time imaging.

Real-time MRI acquisition, reconstruction and visualization is ofparamount importance for diagnostic and therapeutic guidance.Interventional and image-guided therapies as well as interactivediagnostic tasks need rapid image preparation within a few milliseconds.This is hindered, however, by the slow acquisition process, takingseveral minutes to acquire clinically acceptable images. Inefficientacquisition becomes more pronounced for high-resolution and volumetricimages. One possible solution is to decrease the scan duration throughsignificant undersampling. However, such undersampling leads to aseriously ill-posed linear inverse reconstruction problem.

To render the MRI reconstruction well-posed, conventional compressedsensing (CS) incorporates the prior about the inherent lowdimensionality of images by means of sparsity regularization in a propertransform domain such as Wavelet (WV), or, finite differences (or TotalVariation, TV). This typically demands running iterative algorithms, forsolving non-smooth optimization programs, that are time and resourceintensive, and thus not affordable for real-time MRI visualization.Moreover, the sparsity assumption is rather universal and although it isuseful for certain image types, it is oblivious to the inherent latentstructures that are specific to each dataset.

A few attempts have been recently carried out to speed up medical imagereconstruction by leveraging historical patient data, e.g., by traininga network that learns the relation map between an initial aliased imageand the gold-standard one. Although reconstruction speeds up, thesetechniques suffer from blurring and aliasing artifacts. This is mainlybecause they adopt a pixel-wise l₁/l₂ cost for training that isoblivious to structured artifacts and high-frequency texture details.These details, however, are crucial for accurately making diagnosticdecisions. In addition, these techniques lack any mechanism that ensuresthe retrieved images are consistent with the measurements.

Deep neural networks have been used to learn image prior or sparsifyingtransform from historical data in order to solve a nonlinear systemusing iterative optimization algorithms as in the conventional CSmethods. While improving the reconstruction performance, these methodsincur high computational cost due to several iterations for finding theoptimal reconstruction.

BRIEF SUMMARY OF THE INVENTION

To cope with these challenges, the present disclosure provides a CSframework that applies generative adversarial networks (GAN) to modelinga (low-dimensional) manifold of high-quality MR images. Leveraging amixture of least-squares (LS) GANs and pixel-wise l₁/l₂ cost, a deepresidual network with skip connections is trained as the generator thatlearns to remove the aliasing artifacts by projecting onto the imagemanifold. This least-squares generative adversarial network (LSGAN)learns the texture details, while l₁/l₂ suppresses the high-frequencynoise. A discriminator network, which is a multilayer convolutionalneural network (CNN), plays the role of a perceptual cost that is thenjointly trained based on high quality MR images to score the quality ofretrieved images.

In the operational phase, an initial aliased estimate (e.g., simplyobtained by zero-filling) is propagated into the trained generator tooutput the desired reconstructions, that demands very low computationaloverheads.

Extensive evaluations on a large contrast-enhanced MR dataset of imagesrated by expert radiologists corroborate that this generativeadversarial networks compressed sensing (GANCS) technique recovershigher quality images with fine texture details relative to conventionalCS schemes as well as pixel-wise training schemes. In addition, GANCSperforms reconstruction under a few milliseconds, m which is two ordersof magnitude faster than state-of-the-art CS-MRI schemes. Moreover, theperceptual quality metric offered by the trained discriminator networkcan significantly facilitate the radiologists' quality assurance tasks.

In other contexts, generative adversarial networks (GANs) have provensuccessful in modeling distributions (low-dimensional manifolds) andgenerating natural images (high-dimensional data) that are perceptuallyappealing. Despite the success of GANs for local image restoration suchas super-resolution and inpainting, due to fundamental differences, GANshave not been considered for correcting the aliasing artifacts inbiomedical image reconstruction tasks. In essence, aliasing artifacts(e.g., in MRI) emanate from data undersampling in the frequency domainthat globally impacts the entire space domain image. Nevertheless, thepresent approach uses GANs for MRI reconstruction. The approach usesGANs for modeling low-dimensional manifold of high-quality MR images.The images lying on the manifold are not however necessarily consistentwith the observed (undersampled) data. As a result, the reconstructiondeals with modeling the intersection of the image manifold and subspaceof data-consistent images; such a space is an affine subspace for linearmeasurements. To this end, the present GANCS approach adopts a tandemnetwork of a generator (G), an affine projection operator (A), and adiscriminator (D). The generator aims to create gold-standard imagesfrom the complex-valued aliased inputs using a deep residual network(ResNet) with skip connections which retain high resolution information.The data-consistency projection builds upon the (known) signal model andperforms an affine projection onto the space of data consistent images.The D network is a multilayer convolutional neural network (CNN) that istrained using both the “fake” images created by G, and the correspondinggold-standard ones, and aims to correctly distinguish fake from real.Least-squares GANs (LSGANs) is used due to their stability properties.Alternatively, other GANs frameworks may be used, where usually thedifferences are just cost functions, e.g., Wasserstein GAN (W-GAN),Cycle-GAN, BigGAN, or StarGAN. To control the high-frequency texturedetails returned by LSGANs, and to further improve the trainingstability, we partially use the pixel-wise l₁ and l₂ costs for trainingthe generator.

The GANCS results have almost similar quality to the gold-standard(fully-sampled) images, and are superior in terms of diagnostic qualityrelative to the existing alternatives including conventional iterativeCS and deep learning based methods that solely adopt the pixel-wisel₂-based and l₁-based criteria. Moreover, the reconstruction only takesaround 30 ms, which is two orders of magnitude faster thanstate-of-the-art conventional CS toolboxes.

In one aspect, the invention provides a method for diagnostic imagingcomprising: measuring undersampled data y with a diagnostic imagingapparatus; linearly transforming the undersampled data y to obtain aninitial image estimate {tilde over (x)}; applying the initial imageestimate {tilde over (x)} as input to a generator network to obtain analiasing artifact-reduced image x̆ as output of the generator network,wherein the aliasing artifact-reduced image x̆ is a projection onto amanifold of realistic images of the initial image estimate {tilde over(x)}; and performing an acquisition signal model projection of thealiasing artifact-reduced image x̆ onto a space of consistent images toobtain a reconstructed image {circumflex over (x)} having suppressedimage artifacts.

The diagnostic imaging apparatus may be, for example, an MRI scanner,and the undersampled data is k-space data.

Linearly transforming the undersampled data y may comprise, for example,zero padding the undersampled data y, or finding an approximatezero-filling reconstruction from the undersampled data y.

Preferably, the generator network is trained to learn the projectiononto the manifold of realistic images using a set of training images Xand corresponding set of undersampled measurements Y using least-squaresgenerative adversarial network techniques in tandem with a discriminatornetwork to learn texture details and supervised cost function to controlhigh-frequency noise.

The supervised cost function may comprise a mixture of smooth l₂ costand non-smooth l₁ cost.

The discriminator network may be a multilayer deep convolutional neuralnetwork.

The discriminator network may be trained using least-squares cost for adiscriminator decision.

The generator network may be a deep residual network with skipconnections.

In some embodiments, performing the acquisition signal model projectionis implemented as part of the generator network using a softleast-squares penalty during training of the generator network.

In some embodiments, the reconstructed image {circumflex over (x)} isapplied to the generator network to obtain a second aliasingartifact-reduced image, and the second aliasing artifact-reduced imageis projected onto the space of consistent images to obtain a finalreconstructed image.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a flowchart illustrating a method for image reconstructionaccording to an embodiment of the invention.

FIG. 2 is a schematic block diagram illustrating a Generator ResNetarchitecture with residual blocks (RB) according to an embodiment of theinvention.

FIG. 3 is a schematic block diagram illustrating a Discriminatormultilayer CNN architecture according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Consider a generic MRI acquisition model that forms an image x∈R^(N)from k-space projection data y∈R^(M)y=A(x)+v   (1)

where the (possibly) nonlinear map A: C^(N)→C^(M) encompasses theeffects of sampling, coil sensitivities, and the discrete Fouriertransform (DFT). The error term v∈R^(M) captures the noise and unmodeleddynamics. We assume the unknown (complex-valued) image x lies in alow-dimensional manifold, M. No information is known about the manifoldbesides the K training samples X={x_(k)} drawn from it with thecorresponding K (possibly) noisy observations Y={y_(k)}. The data {X,Y}can be obtained for instance from the K past patients in the datasetthat have been already scanned for a sufficient time, and theirhigh-quality reconstruction is available. Given the training data {X,Y},the reconstruction goal is to quickly recover the image x aftercollecting the undersampled measurements y. A flowchart illustrating thesteps of the m reconstruction method is shown in FIG. 1., where step 100performs an MRI acquisition to obtain the under-sampled k-spacemeasurement data 102.

Instead of relying on simple sparsity assumption of X, the approach isto automate the image recovery by learning the nonlinear inversion mapx=A⁻¹(y) from the historical training data {X,Y}. To this end, we beginwith an initial image estimate 106, denoted {tilde over (x)}, that iscalculated in step 104 by a linear transform from undersampledmeasurements y and possibly contains aliasing artifacts. The initialimage estimate {tilde over (x)}=A^(†)(y) may be obtained viazero-filling the missing k-space components, which is the least squaresolution for data-consistency, and running a single iteration ofconjugate gradient. The subsequent reconstruction can then be envisionedas artifact suppression that is modelled as projection onto the manifoldof high-quality images. Learning the corresponding manifold isaccomplished via generative adversarial networks.

The inverse imaging problem is to find a solution at the intersection ofa subspace defined by the acquisition model and the image manifold. Inorder to effectively learn the image manifold from the available(limited number of) training samples, the technique must ensure thetrained manifold contains plausible MR images and must ensure the pointson the manifold are data consistent, i.e., y≈A(x), ∀x∈M.

Alternating Projection with GANs for Plausible Reconstruction To ensureplausibility of the reconstruction, we use GANs. Standard GAN includes atandem network of G and D networks. The initial image estimate {tildeover (x)}=A†(y) is applied as the input to the G network 108. The Gnetwork then projects {tilde over (x)} onto the low-dimensional manifoldM containing the high-quality images X. Let x̆ denote the output 110 ofG. As will be clear later, the G net 108 is trained to learn to projectto the low-dimensional manifold and achieve realistic reconstruction.

Affine Projection and Soft Penalty for Data-Consistency

The output 110 of G may not be consistent with the data. To tackle thisissue, G is followed by another layer 112 that projects the output x̆ ofG onto the set of data-consistent images, namely C={x:y≈A(x)} to obtaina reconstructed image 114, denoted {circumflex over (x)}. For Cartesiangrid with the linear acquisition model y=Ax, the projection isexpressible as {circumflex over (x)}=A^(†)y+P_(N)x̆, whereP_(N)=(I−A^(†)A) resembles projection onto the null space of A.Alternatively, one can impose data consistency to the output of Gthrough a soft least-squares (LS) penalty when training the G network,as will be seen later.

To further ensure that the reconstructed image {circumflex over (x)}falls in the intersection of the manifold M and the set ofdata-consistent images C, we can perform multiple back-and-forthprojections. The network structure in FIG. 1 can then be extended byrepeating the G network and P_(C)(⋅) in serial for a few times, oriterating steps 108 and 112 with feedback of reconstructed image{circumflex over (x)}, as indicated by the dotted line in the figure.For simplicity of exposition, we discuss here a single back-and-forthprojection. However, repeating with multiple projections showssignificant performance improvement.

During training, the final reconstructed image 116 passes through thediscriminator network 118 that tries to output one if {circumflex over(x)}∈X, and zero otherwise 122. The G net 108 learns realisticreconstruction, such that D net 118 cannot always perfectly assign theright labels 122 to the real (fully-sampled) image 120 and “fake”(recovered) images 116.

Least-Squares GANs for Stable Training

Training the networks amounts to playing a game with conflictingobjectives between the generator G and the discriminator D. The Dnetwork aims to score one for the real gold-standard images x, and zerofor the fake/reconstructed images {circumflex over (x)} reconstructed byG. On the other hand, the G network also aims to map the input aliasedimage {tilde over (x)} to a fake image x̆ that looks so realistic andplausible that it can fool D. Various strategies to reach theequilibrium mostly differ in terms of the cost function adopted for Gand D networks. A standard GAN uses a sigmoid cross-entropy loss thatleads to vanishing gradients which renders the training unstable, and asa result it suffers from severe degrees of mode collapse. In addition,for the generated images classified as the real with high confidence(i.e., with large decision variable), no penalty is incurred. Hence, thestandard GAN tends to pull samples away from the decision boundary,which can introduce non-realistic images. Such images can hallucinateimage features, and thus are not reliable for diagnostic decisions. Thepresent method adopts an LS cost for the discriminator decision. Inessence, the LS cost penalizes the decision variables without anynonlinear transformation, and as a result it tends to pull the generatedsamples toward the decision boundary.

Mixed Costs to Avoid High Frequency Noise

One issue with GANs is that they may overemphasize high-frequencytexture, and thus ignore important diagnostic image content. In order todiscard the high-frequency noise and avoid hallucination, the G net ispreferably trained using a supervised l₁/l₂ cost as well. Such mixturewith pixel-wise costs can properly penalize the noise and stabilize thetraining. In particular, the smooth l₂-cost preserves the main structureand leads to a stable training at the expense of introducing blurringartifacts. The non-smooth l₁-cost however may not be as stable as l₂ intraining, but it can better discard the low-intensity noise and achievebetter solutions. All in all, to reveal fine texture details whilediscarding noise, a mixture of LSGAN and l₁/l₂ cost is preferably usedto train the generator. The overall procedure aims to jointly minimizethe expected discriminator costmin_(Θd)E_(x)[(1−D(x;Θ_(d)))²]+E_(y)[D(G({tilde over(x)};Θ_(g));Θ_(d))²],   (P1.1)

where Θ_(d) and Θ_(g) are parameters of the discriminator network D, andgenerator network G, respectively, and the minimum is taken over Θ_(d),and the expected generator costmin_(Θg)E_(y)[∥y−AG({tilde over (x)};Θ_(g))Θ²]+ηE_(x,y)[∥x−G({tilde over(x)};Θ_(g))∥_(1,2)]+λE_(y)[(1−D(G({tilde over(x)};Θ_(g));Θ_(d)))²]  (P1.2)

where the minimum is taken over Θ_(g), and E[⋅] is the statisticalexpectation operator, and ∥⋅∥_(1,2) denotes a convex combination of theelement-wise l₁-norm and l₂-norm with non-negative weights η₁ and η₂respectively, such that η₁+η₂=η. The parameters Θ_(d) and Θ_(g) areusually weights in the CNNs and are trained based on the dataset byoptimizing the above and below cost functions. Usually back-projectionis used. The first LS data fidelity term in (P1.2) is also a softpenalty to ensure the direct output of G network is approximately dataconsistent as mentioned before. Tuning parameters λ and η also controlthe balance between manifold projection, noise suppression, and dataconsistency.

Using the cost (P1.2), taking initial estimation {tilde over (x)} asinput, the generator reconstructs improved x̆=G({tilde over (x)};Θ_(g))from k-space measurement y using the expected regularized-LS estimator,where the regularization is not based on sparsity but learned fromtraining data via LSGAN and l₁-net. Different from the conventional CSscheme, which involves an iterative optimization algorithm to solve forthe l₁/l₂-regularized LS cost, the optimization only happens in trainingand the optimized weights in the network can generalize to any futuresamples. The learned generator can be immediately applied to new testdata to retrieve the image in real time. Even in the presence of LS dataconsistency and l₁/l₂ penalty, the distribution achieved by G networkcan coincide with the true data distribution, which ensures thereconstruction is regularized to be as designed for this manifoldlearning scheme: both data consistent and MRI realistic.

Network Architecture for GANCS

The architectures of G and D nets are now described in relation to FIG.2 and FIG. 3.

Residual Networks for the Generator

The input 200 and output 216 of the generator G are complex-valuedimages of the same size, where the real and imaginary components areconsidered as two separate channels. The input image {tilde over (x)} issimply an initial estimate obtained, e.g., via zero-filling, whichundergoes aliasing artifacts. After convolving the input channels withdifferent kernels, they are added up in the next layer. All networkkernels are assumed real-valued. A deep residual network (ResNet) isused for the generator that contains 5 residual blocks 202 through 204.As shown in the detail 218 for block 202, each block has twoconvolutional layers with small 3×3 kernels and 128 feature maps thatare followed by batch normalization (BN) and rectified linear unit(ReLU) activation. The five residual blocks are followed by threeconvolutional layers with map size 1×1. The first layer has aconvolution 206 and ReLU activation 208. Similarly, the second layer hasa convolution 210 and ReLU activation 212, while the last layer hasconvolution 214 but uses no activation to return two output channelscorresponding the real and imaginary image parts.

The G network learns the projection onto the manifold of realistic MRimages. The manifold dimension is controlled by the number of residualblocks (RB), feature maps, stride size, and the size of discriminator.In the figure, n and k refer to number of feature maps and filter size,respectively.

Convolutional Neural Networks for Discriminator

The D network takes the magnitude of the complex-valued output of the Gnet and data consistency projection as an input 300. In a preferredembodiment, it is composed of a series of convolutional layers, where inall layers except the last one, a convolution operation is followed bybatch normalization, and subsequently by ReLU activation. No pooling isused. Layer 1 has a convolution 302 followed by batch normalization 304and ReLU activation 306. Similarly, layer 4 has a convolution 308followed by batch normalization 310 and ReLU activation 312, and layer 5has a convolution 314 followed by batch normalization 316 and ReLUactivation 318. Layer 6 is a convolution 320 only, and layer 7 is anaverage 322. For the first four layers, the number of feature maps isdoubled at each successive layer from 8 to 64, while at the same timeconvolution with stride 2 is used to reduce the image resolution. Kernelsize 3×3 is adopted for the first 4 layers, while layers 5 and 6 usekernel size 1×1. The last layer 7 simply averages out the features toform the decision variable for binary classification. No soft-maxoperation is used. The variables n, k, and s in the figure refer tonumber of feature maps, filter size, and stride size, respectively.

Evaluations

Effectiveness of the GANCS scheme was assessed in this section for asingle-coil MR acquisition model with Cartesian sampling. For the n-thpatient, the acquired k-space data is denoted y_(i,j)^((n))=[F(X_(n))]_(i,j)+v_(i,j) ^((n)), where (i,j)∈Ω. We adopt thesingle coil model for demonstration purposes, but extension tomulti-coil MRI acquisition model is straightforward by simply updatingthe signal model. Sampling set Ω indexes the sampled Fouriercoefficients. As it is conventionally performed with CS MRI, we select Ωbased on a variable density sampling with radial view ordering thattends to sample more low frequency components from the center ofk-space. Different undersampling rates (5 and 10) are chosen throughoutthe experiment. The input zero-filling (ZF) image {tilde over (x)} issimply generated using inverse 2D FT of the sampled k-space, which isseverely contaminated by aliasing artifacts. Input images are normalizedto have the maximum magnitude unity per image.

Adam optimizer is used with the momentum parameter β=0.9, mini-batchsize 4, and initial learning rate 10-6 that is halved every 10,000iterations. Training is performed with TensorFlow interface on a NVIDIATitan X Pascal GPU, 12 GB RAM. We allow 20 epochs that takes around 10hours for training.

For the dataset, abdominal image volumes were acquired for 350 pediatricpatients after gadolinium-based contrast enhancement. Each 3D volumeincludes from 150 to 220 axial slices of size 256×128 with voxelresolution 1.07×1.12×2.4 mm. Axial slices are used as input images fortraining a neural network. 340 patients (50,000 2D slices) areconsidered for training, and 10 patients (1,920 2D slices) for testing.All scans were acquired on a 3T MRI scanner (GE MR750).

Training Convergence

Stable convergence of GANCS was confirmed by considering evolution ofdifferent components of G and D costs for training over batches (size4), with η=0.025 and λ=0.975 as an example to emphasize the GAN loss intraining. According to (P1.2), the G cost mainly pertains to the lastterm which shows how well the G net can fool the D net. The D cost alsoincludes two components based m on (P1.1) associated with theclassification performance for both real and fake images. It wasconfirmed that all cost components decrease, and after about 5,000batches it reaches the equilibrium cost 0.25. This implies that uponconvergence the G-net images become so realistic that the D-net willbehave as a flipping coin, i.e., D({circumflex over (x)})=½. In thissetting with a hard affine projection layer no data-consistency cost isincurred.

It is also worth mentioning that to improve the convergence stability ofGANCS, and to ensure the initial distributions of fake and real imagesare overlapping, we trained with pure l₁ cost (η=1, λ=0) at thebeginning and then gradually switch to the mixture loss intended.

Quantitative Image Evaluation and Comparison

For comparison, images were reconstructed by various methods with 5-foldand 10-fold undersampling of k-space. Specifically, the gold-standardimage, was compared with images reconstructed by GANCS with l₁-cost(η=0.975, λ=0.025), GANCS with l₁-cost (η=1, λ=0), GANCS with l₂-cost(η=1, λ=0), and CS-WV. For 5-fold undersampling, the ZF reconstructionis also included. CS reconstruction is performed using the BerkeleyAdvanced Reconstruction Toolbox (BART), where the tuning parameters areoptimized for the best performance. These image comparisons confirmedthat GANCS with l₁ cost (η=0.975, λ=0.025) returns the sharpest imageswith highest contrast and texture details that can reveal the smallanatomical details. Images retrieved by GANCS with l₂-cost alone resultsin overly smooth textures as the l₂-cost encourages finding pixel-wiseaverages of all plausible solutions. Also, images obtained using GANCSwith l₁ alone look more realistic than the l₂ counterpart. Thereconstructed images however are not as sharp as the GANCS (η=0.975,λ=0.025) which leverages both l₁-net and GANs. We have observed thatusing m GAN alone (η=0, λ=1), the retrieved images are quite sharp witha high-frequency noise present over the image that can distort the imagestructure. It turns out that including the l₁ cost during trainingbehaves as a low-pass filter to discard the high-frequency noises, whilestill achieving reasonably sharp images. It is also evident that CS-WVintroduces blurring artifacts. We also tested CS-TV, but CS-WV isobserved to consistently outperform CS-TV, and thus we choose CS-WV asthe representative for CS-MRI.

Reconstructing 30 slices per second makes GANCS a suitable choice forreal-time imaging. In terms of SNR and SSIM, GANCS with l₁-cost aloneachieves the best performance. GANCS with proper l₁-cost mixing canachieve good performance with a marginally decrease from GANCS withl₁-cost alone.

Diagnostic Quality Assessment

The perceptual quality of resulting images was confirmed by radiologistopinion scores (ROS). The images retrieved by GANCS attain the highestscore that is as good as the gold-standard.

Performance on Abnormal Cases

To address the concern that GANCS may create hallucinated images, twoabnormal patients with missing left and right kidneys were scanned andimages reconstructed, where the training data does not include patientswith similar abnormalities. It was confirmed that GANCSmisses/introduces no structures or edges.

Number of Patients for Prediction

Prediction (generalization) performance of the deep learning modelheavily depends on the amount of training data. This becomes moreimportant when dealing with scarce medical data that are typically notaccessible in large scales due to privacy concerns and institutionalregulations. To address this question we examined an evaluation scenarioto assess the reconstruction performance for a fixed test dataset, forvariable number of patients used for training. The test measured SNRversus the number of training patients for the GANCS scheme withη=0.975, λ=0.025. As the number of patients increased from 1 to 130, anoticeable SNR gain was observed. The performance gain then graduallysaturates as the number of patients reaches 150. It thus seems with 150or more patients we can take full advantage of both learning fromhistorical data and the complexity of the networks. Recall that a fixedsampling mask is used for training and testing. GANCS, however, capturesthe signal model, and therefore it can easily accommodate differentsampling trajectories. Also note that, if more datasets are availablefor training, we can further improve the model performance by increasingmodel complexity. Further study of the number of patients needed forother random sampling schemes and different network models is animportant question that is a focus of our current research.

Discriminator Interpretation

As suggested by the training strategy, the discriminator plays a rolelike a radiologist that scores the quality of images created by thegenerator. During adversarial training, D learns to correctly discernthe real fully-sampled images from the fake ones, where the fake onesbecome quite realistic as training progresses. It is thus insightful tounderstand image features that drive the quality score. To this end, wecompared original images with heat maps of feature maps of D net athidden convolutional layers. This demonstrated that, after learning fromtens of thousands of generated MRI images by the G network together withthe corresponding gold-standard ones, where different organs are mpresent, the D network learns to focus on certain regions of interestthat are more susceptible to artifacts.

CONCLUSIONS

A CS framework is provided that leverages the historical data for rapidand high diagnostic-quality image reconstruction from highlyundersampled MR measurements. A low-dimensional manifold is learnedwhere the reconstructed images have not only superior sharpness anddiagnostic quality, but also consistent with both the real MRI data andthe acquisition model. To this end, a neural network scheme based onLSGANs and l₁/l₂ costs is trained, where a generator is used to map areadily obtainable undersampled image to a realistic-looking oneconsistent with the measurements, while a discriminator network istrained jointly to score the quality of the resulting image. The overalltraining acts as a game between generator and discriminator that makesthem more intelligent at reconstruction and quality evaluation.

The invention claimed is:
 1. A method for diagnostic imaging comprising:measuring undersampled data y with a diagnostic imaging apparatus;linearly transforming the undersampled data y to obtain an initial imageestimate {tilde over (x)}; applying the initial image estimate {tildeover (x)} as input to a generator network to obtain an aliasingartifact-reduced image x̆ as output of the generator network, wherein thealiasing artifact-reduced image x̆ is a projection onto a manifold ofrealistic images of the initial image estimate {tilde over (x)};performing an acquisition signal model projection of the aliasingartifact-reduced image x̆ onto a space of consistent images to obtain areconstructed image {circumflex over (x)} having suppressed imageartifacts.
 2. The method of claim 1 wherein the diagnostic imagingapparatus is an MRI scanner, and wherein the undersampled data isk-space data.
 3. The method of claim 1 wherein linearly transforming theundersampled data y comprises zero padding the undersampled data y. 4.The method of claim 1 wherein linearly transforming the undersampleddata y comprises finding an approximate zero-filling reconstruction fromthe undersampled data y.
 5. The method of claim 1 wherein the generatornetwork is trained to learn the projection onto the manifold ofrealistic images using a set of training images X and corresponding setof undersampled measurements Y using least-squares generativeadversarial network techniques in tandem with a discriminator network tolearn texture details and supervised cost function to controlhigh-frequency noise.
 6. The method of claim 5 wherein the supervisedcost function comprises a mixture of smooth l₂ cost and non-smooth l₁cost.
 7. The method of claim 5 wherein the discriminator network is amultilayer deep convolutional neural network.
 8. The method of claim 5wherein the discriminator network is trained using a least squares costfor a discriminator decision.
 9. The method of claim 1 wherein thegenerator network is a deep residual network with skip connections. 10.The method of claim 1 wherein performing an acquisition signal modelprojection is implemented as part of the generator network using a softleast-squares penalty during training of the generator network.
 11. Themethod of claim 1 wherein the reconstructed image {circumflex over (x)}is applied to the generator network to obtain a second aliasingartifact-reduced image, and the second aliasing artifact-reduced imageis projected onto the space of consistent images to obtain a finalreconstructed image.